MIT 18.S191 Computational Thinking: Introduction to Images

• What is an image, though?
• How can we store an image in the computer?
• Load image using scikit-image (skimage) package
• Slicing and Indexing
• Manipulating matrices
• Manipulating an image
• Painting a piece of an image
  • Using for loops
  • Using element-wise operations via broadcasting
• Modifying the whole image at once
• Transforming an image
  • Blurring

Note: This notebook is adapted from

> MIT 18.S191 Introduction to Computational Thinking course (Fall 2020).> The course uses Julia Programming language, and I> thought it would be fun to follow along while using Python :).

Note: Check out the lecture video on YouTube

What is an image, though?

• A grid of colored squares called pixels
• A color of each pair (i, j) of indices
• A discretization

How can we store an image in the computer?

• A 1D array --> vector
• A 2D array --> matrix
• A 3D array --> tensor

Load image using scikit-image (skimage) package

import skimage.io
import numpy as np
import warnings

warnings.filterwarnings("ignore")

url = "https://i.imgur.com/VGPeJ6s.jpg"
philip = skimage.io.imread(url)
skimage.io.imshow(philip)

type(philip)

numpy.ndarray

philip.dtype

dtype('uint8')

philip.shape

(3675, 2988, 3)

Slicing and Indexing

# Show the 100th and 400th pixels
skimage.io.imshow(philip[100:101, 400:401, :])

skimage.io.imshow(philip[100, 400].reshape(1, 1, 3))

h, w, _ = philip.shape
# Extract bottom half of the image
# and a tenth of the width until nine tenth of the width into the image
head = philip[(h // 2) : h, (w // 10) : ((9 * w) // 10)]
skimage.io.imshow(head)

head.shape

(1838, 2391, 3)

philip.shape

(3675, 2988, 3)

Manipulating matrices

An image is just a matrix, so we can manipulate matrices to manipulate the image:

stacked_philip = np.hstack([head, head])
stacked_philip.shape

(1838, 4782, 3)

skimage.io.imshow(stacked_philip)

a = np.hstack([head, np.flip(head, axis=1)])
b = np.hstack([np.flip(head, axis=0), np.flip(np.flip(head, axis=1), axis=0)])
c = np.vstack([a, b])
a.shape, b.shape, c.shape

((1838, 4782, 3), (1838, 4782, 3), (3676, 4782, 3))

skimage.io.imshow(c)

Manipulating an image

• How can we get inside the image and change it?
• There are two possibilities:
  • Modify or mutate numbers inside the array. This is useful when we want to change a small piece
  • Create a new copy of the array. This is useful when we want to alter everything together

Painting a piece of an image

• Let's paint a corner red
• We'll copy the image first so we don't destroy the original

new_phil = head.copy()
skimage.io.imshow(new_phil)

Using for loops

%%timeit
for i in range(1, 100):
    for j in range(1, 300):
        new_phil[i, j] = [255, 0, 0] # Paint this corner red

45.7 ms ± 4.64 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

skimage.io.imshow(new_phil)

Using element-wise operations via broadcasting

Instead of using for loops, it is better and more efficient to let NumPy do the broadcasting for us during arithmetic operations.

See NumPy broadcasting documentation for detailed explanation

new_phil2 = new_phil.copy()

%%timeit
new_phil2[100:200, 1:100] = [0, 255, 0] # Paint this corner green

54.2 µs ± 3.32 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

skimage.io.imshow(new_phil2)

Modifying the whole image at once

• We can use the same broadcasting trick to modify the whole image at once
• Let's redify the image

def redify(image):
    """Function that turns a color into just its red component"""
    img = image.copy()
    img[:, :, 1:] = 0
    return img


skimage.io.imshow(redify(new_phil))

Transforming an image

• Let's decimate (downsample) philip original image

def decimate(img, r):
    """Pull every other `r` element in the array"""
    return img[0::r, 0::r]

poor_phil = decimate(philip, 5)
poor_phil.shape

(735, 598, 3)

skimage.io.imshow(poor_phil)

Blurring

import skimage.filters
import matplotlib.pyplot as plt
import panel as pn

# Apply Gaussian blur, creating a new image
def blur(image, sigma):
    blurred_phil = skimage.filters.gaussian(
        image, sigma=(sigma, sigma), multichannel=True
    )
    return blurred_phil


# Create a widget for exploring different sigma values
sigma = pn.widgets.Player(
    name="Sigma", start=1, end=30, value=1, loop_policy="once"
)


@pn.depends(sigma)
def viewer(sigma):
    img = blur(image=head, sigma=sigma)
    ax = skimage.io.imshow(img)
    plt.title(f"Sigma={sigma}")
    fig = ax.get_figure()
    plt.close(fig)
    return fig


p = pn.Column("<br>\n# Image Viewer\n**Select sigma value**", sigma, viewer)
p.save("apps/blur_explorer.html", embed=True)

 72%|███████▏  | 21/29 [01:14<00:20,  2.55s/it]

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

 86%|████████▌ | 25/29 [01:23<00:08,  2.25s/it]

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

 93%|█████████▎| 27/29 [01:27<00:04,  2.12s/it]

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

from IPython.display import HTML, display, display_html

display(HTML("apps/blur_explorer.html"))

<!DOCTYPE html>

%load_ext watermark
%watermark -d -iv -m -g

panel   0.9.7
skimage 0.17.2
numpy   1.19.1
2020-09-12

compiler   : Clang 10.0.1
system     : Darwin
release    : 18.7.0
machine    : x86_64
processor  : i386
CPU cores  : 8
interpreter: 64bit
Git hash   : 70425fe2369d2472a93365b27aa000c684328009